151 research outputs found
Emergence of Human-comparable Balancing Behaviors by Deep Reinforcement Learning
This paper presents a hierarchical framework based on deep reinforcement
learning that learns a diversity of policies for humanoid balance control.
Conventional zero moment point based controllers perform limited actions during
under-actuation, whereas the proposed framework can perform human-like
balancing behaviors such as active push-off of ankles. The learning is done
through the design of an explainable reward based on physical constraints. The
simulated results are presented and analyzed. The successful emergence of
human-like behaviors through deep reinforcement learning proves the feasibility
of using an AI-based approach for learning humanoid balancing control in a
unified framework
Learning dynamic motor skills for terrestrial locomotion
The use of Deep Reinforcement Learning (DRL) has received significantly increased attention
from researchers within the robotics field following the success of AlphaGo, which demonstrated
the superhuman capabilities of deep reinforcement algorithms in terms of solving complex
tasks by beating professional GO players. Since then, an increasing number of researchers
have investigated the potential of using DRL to solve complex high-dimensional robotic tasks,
such as legged locomotion, arm manipulation, and grasping, which are difficult tasks to solve
using conventional optimization approaches.
Understanding and recreating various modes of terrestrial locomotion has been of long-standing interest to roboticists. A large variety of applications, such as rescue missions,
disaster responses and science expeditions, strongly demand mobility and versatility in legged
locomotion to enable task completion. In order to create useful physical robots, it is necessary
to design controllers to synthesize the complex locomotion behaviours observed in humans
and other animals.
In the past, legged locomotion was mainly achieved via analytical engineering approaches.
However, conventional analytical approaches have their limitations, as they require relatively
large amounts of human effort and knowledge. Machine learning approaches, such as DRL,
require less human effort compared to analytical approaches. The project conducted for this
thesis explores the feasibility of using DRL to acquire control policies comparable to, or better
than, those acquired through analytical approaches while requiring less human effort.
In this doctoral thesis, we developed a Multi-Expert Learning Architecture (MELA) that
uses DRL to learn multi-skill control policies capable of synthesizing a diverse set of dynamic
locomotion behaviours for legged robots. We first proposed a novel DRL framework for the
locomotion of humanoid robots. The proposed learning framework is capable of acquiring
robust and dynamic motor skills for humanoids, including balancing, walking, standing-up
fall recovery. We subsequently improved upon the learning framework and design a novel
multi-expert learning architecture that is capable of fusing multiple motor skills together in
a seamless fashion and ultimately deploy this framework on a real quadrupedal robot. The
successful deployment of learned control policies on a real quadrupedal robot demonstrates
the feasibility of using an Artificial Intelligence (AI) based approach for real robot motion control
Learning Motor Skills of Reactive Reaching and Grasping of Objects
Reactive grasping of objects is an essential capability of autonomous robot manipulation, which is yet challenging to learn such sensorimotor control to coordinate coherent hand-finger motions and be robust against disturbances and failures. This work proposed a deep reinforcement learning based scheme to train feedback control policies which can coordinate reaching and grasping actions in presence of uncertainties. We formulated geometric metrics and task-orientated quantities to design the reward, which enabled efficient exploration of grasping policies. Further, to improve the success rate, we deployed key initial states of difficult hand-finger poses to train policies to overcome potential failures due to challenging configurations. The extensive simulation validations and benchmarks demonstrated that the learned policy was robust to grasp both static and moving objects. Moreover, the policy generated successful failure recoveries within a short time in difficult configurations and was robust with synthetic noises in the state feedback which were unseen during training
Learning Pregrasp Manipulation of Objects from Ungraspable Poses
In robotic grasping, objects are often occluded in ungraspable configurations
such that no pregrasp pose can be found, eg large flat boxes on the table that
can only be grasped from the side. Inspired by humans' bimanual manipulation,
eg one hand to lift up things and the other to grasp, we address this type of
problems by introducing pregrasp manipulation - push and lift actions. We
propose a model-free Deep Reinforcement Learning framework to train control
policies that utilize visual information and proprioceptive states of the robot
to autonomously discover robust pregrasp manipulation. The robot arm learns to
first push the object towards a support surface and establishes a pivot to lift
up one side of the object, thus creating a clearance between the object and the
table for possible grasping solutions. Furthermore, we show the effectiveness
of our proposed learning framework in training robust pregrasp policies that
can directly transfer from simulation to real hardware through suitable design
of training procedures, state, and action space. Lastly, we evaluate the
effectiveness and the generalisation ability of the learned policies in
real-world experiments, and demonstrate pregrasp manipulation of objects with
various size, shape, weight, and surface friction.Comment: 8 pages open access version for ICRA2020 6 pages acceptance pape
Multi-expert learning of adaptive legged locomotion
Achieving versatile robot locomotion requires motor skills which can adapt to
previously unseen situations. We propose a Multi-Expert Learning Architecture
(MELA) that learns to generate adaptive skills from a group of representative
expert skills. During training, MELA is first initialised by a distinct set of
pre-trained experts, each in a separate deep neural network (DNN). Then by
learning the combination of these DNNs using a Gating Neural Network (GNN),
MELA can acquire more specialised experts and transitional skills across
various locomotion modes. During runtime, MELA constantly blends multiple DNNs
and dynamically synthesises a new DNN to produce adaptive behaviours in
response to changing situations. This approach leverages the advantages of
trained expert skills and the fast online synthesis of adaptive policies to
generate responsive motor skills during the changing tasks. Using a unified
MELA framework, we demonstrated successful multi-skill locomotion on a real
quadruped robot that performed coherent trotting, steering, and fall recovery
autonomously, and showed the merit of multi-expert learning generating
behaviours which can adapt to unseen scenarios
Unsupervised Deep Cross-Language Entity Alignment
Cross-lingual entity alignment is the task of finding the same semantic
entities from different language knowledge graphs. In this paper, we propose a
simple and novel unsupervised method for cross-language entity alignment. We
utilize the deep learning multi-language encoder combined with a machine
translator to encode knowledge graph text, which reduces the reliance on label
data. Unlike traditional methods that only emphasize global or local alignment,
our method simultaneously considers both alignment strategies. We first view
the alignment task as a bipartite matching problem and then adopt the
re-exchanging idea to accomplish alignment. Compared with the traditional
bipartite matching algorithm that only gives one optimal solution, our
algorithm generates ranked matching results which enabled many potentials
downstream tasks. Additionally, our method can adapt two different types of
optimization (minimal and maximal) in the bipartite matching process, which
provides more flexibility. Our evaluation shows, we each scored 0.966, 0.990,
and 0.996 Hits@1 rates on the DBP15K dataset in Chinese, Japanese, and French
to English alignment tasks. We outperformed the state-of-the-art method in
unsupervised and semi-supervised categories. Compared with the state-of-the-art
supervised method, our method outperforms 2.6% and 0.4% in Ja-En and Fr-En
alignment tasks while marginally lower by 0.2% in the Zh-En alignment task.Comment: 17 pages,5 figures, Accepted by ECML PKDD 2023(Research Track
Learning natural locomotion behaviors for humanoid robots using human bias
This paper presents a new learning framework that leverages the knowledge
from imitation learning, deep reinforcement learning, and control theories to
achieve human-style locomotion that is natural, dynamic, and robust for
humanoids. We proposed novel approaches to introduce human bias, i.e. motion
capture data and a special Multi-Expert network structure. We used the
Multi-Expert network structure to smoothly blend behavioral features, and used
the augmented reward design for the task and imitation rewards. Our reward
design is composable, tunable, and explainable by using fundamental concepts
from conventional humanoid control. We rigorously validated and benchmarked the
learning framework which consistently produced robust locomotion behaviors in
various test scenarios. Further, we demonstrated the capability of learning
robust and versatile policies in the presence of disturbances, such as terrain
irregularities and external pushes.Comment: university polic
- …